3 research outputs found
Using Mobile Data and Deep Models to Assess Auditory Verbal Hallucinations
Hallucination is an apparent perception in the absence of real external
sensory stimuli. An auditory hallucination is a perception of hearing sounds
that are not real. A common form of auditory hallucination is hearing voices in
the absence of any speakers which is known as Auditory Verbal Hallucination
(AVH). AVH is fragments of the mind's creation that mostly occur in people
diagnosed with mental illnesses such as bipolar disorder and schizophrenia.
Assessing the valence of hallucinated voices (i.e., how negative or positive
voices are) can help measure the severity of a mental illness. We study N=435
individuals, who experience hearing voices, to assess auditory verbal
hallucination. Participants report the valence of voices they hear four times a
day for a month through ecological momentary assessments with questions that
have four answering scales from ``not at all'' to ``extremely''. We collect
these self-reports as the valence supervision of AVH events via a mobile
application. Using the application, participants also record audio diaries to
describe the content of hallucinated voices verbally. In addition, we passively
collect mobile sensing data as contextual signals. We then experiment with how
predictive these linguistic and contextual cues from the audio diary and mobile
sensing data are of an auditory verbal hallucination event. Finally, using
transfer learning and data fusion techniques, we train a neural net model that
predicts the valance of AVH with a performance of 54\% top-1 and 72\% top-2 F1
score
VIMES : A Wearable Memory Assistance System for Automatic Information Retrieval
The advancement of artificial intelligence and wearable computing triggers the radical innovation of cognitive applications. In this work, we propose VIMES, an augmented reality-based memory assistance system that helps recall declarative memory, such as whom the user meets and what they chat. Through a collaborative method with 20 participants, we design VIMES, a system that runs on smartglasses, takes the first-person audio and video as input, and extracts personal profiles and event information to display on the embedded display or a smartphone. We perform an extensive evaluation with 50 participants to show the effectiveness of VIMES for memory recall. VIMES outperforms (90% memory accuracy) other traditional methods such as self-recall (34%) while offering the best memory experience (Vividness, Coherence, and Visual Perspective all score over 4/5). The user study results show that most participants find VIMES useful (3.75/5) and easy to use (3.46/5).Peer reviewe